feat(scoring): act on unmodeled-constant drift (Fixes #811) by kiannidev · Pull Request #832 · JSONbored/loopover

kiannidev · 2026-06-17T15:21:38Z

Summary

Route findUnmodeledUpstreamConstants output into the existing Pipeline-B upstream drift surface via a stable-fingerprint report (scoring_model / unmodeled_scoring_constants).
Sync on both refreshScoringModelSnapshot and refreshUpstreamDrift, resolving the report when all upstream constants are modeled.
Existing fileUpstreamDriftIssues + GITTENSORY_AUTO_FILE_DRIFT_ISSUES can now auto-file GitHub issues for this signal.

Test plan

npx vitest run test/unit/unmodeled-scoring-drift.test.ts
npx vitest run test/unit/scoring.test.ts test/unit/upstream-ruleset.test.ts
With GITTENSORY_AUTO_FILE_DRIFT_ISSUES=true, confirm scheduled file-upstream-drift-issues creates/updates an issue when upstream adds an unmodeled constant

Fixes #811

Persist a stable upstream drift report when gittensor defines scoring constants gittensory does not model, so scheduled issue filing and operator status can act on the signal instead of burying it in snapshot warnings alone. Fixes JSONbored#811 Co-authored-by: Cursor <cursoragent@cursor.com>

ghost · 2026-06-17T15:21:54Z

Note

Gittensory Gate skipped

PR closed before full evaluation. No late first comment was created.

Signal	Result	Evidence	Action
Gate result	⚠️ Skipped	#832 is no longer open.	No action.

💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

superagent-security · 2026-06-17T15:22:47Z

Superagent didn't find any vulnerabilities or security issues in this PR.

ghost · 2026-06-17T15:23:23Z

gittensory · advisory review

Reviewed 6 changed file(s) — two independent AI reviewers.

Changed files: src/scoring/model.ts, src/upstream/ruleset.ts, src/upstream/unmodeled-scoring-drift.ts, test/unit/scoring.test.ts, test/unit/unmodeled-scoring-drift.test.ts, test/unit/upstream-ruleset.test.ts

Suggested action: ✅ Safe to merge — both reviewers found no blocking issues.

Reviewer A · gpt-oss-120b — recommends ✅ merge
The PR introduces drift detection for unmodeled scoring constants, adds a new utility module, integrates it into scoring model refresh and upstream drift refresh, and provides comprehensive tests. The implementation follows existing patterns and respects the public/private sanitization rules.

Suggestions

Document the new findUnmodeledConstantKeys export in the scoring model module README or JSDoc for future maintainers.
Consider adding a unit test that simulates a DB failure during upsertUpstreamDriftReport to verify graceful error handling.
Verify that the Node test environment includes crypto.randomUUID; if not, polyfill or import from the crypto module.

Worth double-checking

Performance impact of scanning up to 50 drift reports on each refresh; monitor if this becomes a bottleneck.
Potential future changes to the fingerprint seed must be coordinated to avoid breaking existing reports.

Reviewer B · nemotron-3-120b-a12b — recommends ✅ merge
The PR adds detection and reporting of unmodeled scoring constants from upstream (Gittensor) to improve visibility of drift. It introduces a new drift report type, integrates it into scoring model and ruleset refresh flows, and includes comprehensive unit tests. The changes are cohesive, well-tested, and maintain the public/private boundary (no forbidden terms in drift reports).

Suggestions

Consider adding a comment in src/upstream/unmodeled-scoring-drift.ts explaining why SCORING_MODEL_FOLLOW_UP includes specific files (to aid future maintenance).
In test/unit/unmodeled-scoring-drift.test.ts, the test 'preserves linked issue metadata' could be strengthened by verifying that the issue metadata is preserved when the report is resolved (though current behavior is acceptable).
The hardcoded upstream repo/ref fallbacks ('entrius/gittensor', 'test') should be verified against documentation to ensure they match expected defaults.

Worth double-checking

Duplicate drift report generation: syncUnmodeledScoringConstantDrift is called from both refreshScoringModelSnapshot and refreshUpstreamDrift. While deduplicated by fingerprint, ensure this doesn't cause unnecessary writes during frequent runs.
The severity threshold (>=3 constants = high) is reasonable but should be revisited if false positives arise; consider making it configurable via env in future.
Ensure the drift report's 'recommendedFollowUp' array stays accurate if file paths change (though this is low-risk as it's advisory).

codecov · 2026-06-17T15:23:26Z

Codecov Report

❌ Patch coverage is 97.14286% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 96.71%. Comparing base (e879ffa) to head (31c84fb).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
src/upstream/ruleset.ts	75.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #832   +/-   ##
=======================================
  Coverage   96.71%   96.71%           
=======================================
  Files         108      109    +1     
  Lines       14641    14674   +33     
  Branches     5298     5314   +16     
=======================================
+ Hits        14160    14192   +32     
  Misses        102      102           
- Partials      379      380    +1

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Exercise env fallbacks, resolve/no-op paths, error-source skip, and long constant summaries so changed lines meet the 97% patch coverage requirement. Co-authored-by: Cursor <cursoragent@cursor.com>

superagent-security · 2026-06-18T05:18:49Z

⚠️ Contributor trust inconclusive. Click here for more info: Superagent Dashboard

JSONbored

Good:

Cleanly adds unmodeled upstream scoring-constant drift detection.
Integrates with scoring refresh and upstream drift refresh.
Tests cover create/resolve/severity behavior.
CI, Codecov, Superagent, and Gittensory gate are passing.

Bad:

Minor metadata nit: scoring-model refresh hardcodes entrius/gittensor:test in one source path.

Change requests:

Fix the metadata nit.

Stop hardcoding entrius/gittensor:test in refreshScoringModelSnapshot; let syncUnmodeledScoringConstantDrift resolve repo/ref from env defaults. Addresses review on JSONbored#832. Co-authored-by: Cursor <cursoragent@cursor.com>

kiannidev · 2026-06-18T14:19:54Z

@JSONbored
I addressed feedback.
Please review PR again.

JSONbored

PR #832 — feat(scoring): act on unmodeled-constant drift (Fixes #811) [kiannidev]
Action: APPROVE
Issue #811: FULLY CI: pass
Good:

syncUnmodeledScoringConstantDrift opens/updates/resolves a stable-fingerprint upstream drift report
from both refreshScoringModelSnapshot and refreshUpstreamDrift; severity scales with count.
The maintainer's only nit (hardcoded entrius/gittensor:test) is resolved — source is now env-driven
(GITTENSOR_UPSTREAM_REPO/REF) with a fallback, and a test asserts the override.
9-case unit suite + integration coverage. On-direction backend work.

kiannidev requested a review from JSONbored as a code owner June 17, 2026 15:21

dosubot Bot added the size:L label Jun 17, 2026

ghost added the gittensory:reviewed label Jun 17, 2026

test(scoring): cover unmodeled drift branches for codecov patch gate

2356936

Exercise env fallbacks, resolve/no-op paths, error-source skip, and long constant summaries so changed lines meet the 97% patch coverage requirement. Co-authored-by: Cursor <cursoragent@cursor.com>

JSONbored assigned kiannidev Jun 17, 2026

JSONbored added the gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. label Jun 17, 2026

ghost mentioned this pull request Jun 17, 2026

fix(crypto): reject malformed hex in timingSafeEqualHex #661

Closed

1 task

Merge branch 'main' into feat/issue-811-unmodeled-drift

92d0267

This was referenced Jun 17, 2026

feat(scoring): wire issue-discovery/issue-spam scoring constants into engine #835

Merged

feat(ui): audit feed / skipped-PR viewer #831

Merged

Merge branch 'main' into feat/issue-811-unmodeled-drift

10b7d32

Merge branch 'main' into feat/issue-811-unmodeled-drift

5cff047

JSONbored requested changes Jun 18, 2026

View reviewed changes

kiannidev requested a review from JSONbored June 18, 2026 14:19

Merge branch 'main' into feat/issue-811-unmodeled-drift

97c8c7d

JSONbored approved these changes Jun 18, 2026

View reviewed changes

dosubot Bot added the lgtm label Jun 18, 2026

Merge branch 'main' into feat/issue-811-unmodeled-drift

31c84fb

JSONbored merged commit 3a5e65a into JSONbored:main Jun 18, 2026
17 checks passed

Navigation Menu

Uh oh!

feat(scoring): act on unmodeled-constant drift (Fixes #811) - #832

feat(scoring): act on unmodeled-constant drift (Fixes #811)#832
JSONbored merged 8 commits into
JSONbored:mainfrom
kiannidev:feat/issue-811-unmodeled-drift

kiannidev commented Jun 17, 2026 •

edited

Loading

Uh oh!

ghost commented Jun 17, 2026 •

edited by ghost

Loading

Gittensory Gate skipped

Uh oh!

superagent-security Bot commented Jun 17, 2026

Uh oh!

ghost commented Jun 17, 2026 •

edited by ghost

Loading

Uh oh!

codecov Bot commented Jun 17, 2026 •

edited

Loading

Uh oh!

superagent-security Bot commented Jun 18, 2026

Uh oh!

JSONbored left a comment

Uh oh!

kiannidev commented Jun 18, 2026

Uh oh!

JSONbored left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

kiannidev commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Uh oh!

ghost commented Jun 17, 2026 • edited by ghost Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Gittensory Gate skipped

Uh oh!

superagent-security Bot commented Jun 17, 2026

Uh oh!

ghost commented Jun 17, 2026 • edited by ghost Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov Bot commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

superagent-security Bot commented Jun 18, 2026

Uh oh!

JSONbored left a comment

Choose a reason for hiding this comment

Uh oh!

kiannidev commented Jun 18, 2026

Uh oh!

JSONbored left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

kiannidev commented Jun 17, 2026 •

edited

Loading

ghost commented Jun 17, 2026 •

edited by ghost

Loading

ghost commented Jun 17, 2026 •

edited by ghost

Loading

codecov Bot commented Jun 17, 2026 •

edited

Loading